Search Results: "ianw"

17 November 2014

Daniel Leidert: Removal of debian.wgdd.de and cvs,svn,vcs .wgdd.de

If you've recently tried to browse to or apt-get from either cvs.wgdd.de, svn.wgdd.de, vcs.wgdd.de, debian.wgdd.de or ubuntu.wgdd.deyou've probably seen (and still are) an error (410, Gone) coming up and I'd like to give a short explanation why. cvs,svn,vcs .wgdd.deI've left my server provider and shut down the above services and only keep a small amount of services running. The domains cvs,svn,vcs .wgdd.de were used to provide (a) a subversion (SVN) server (via HTTPS and dav_svn) for some public and private work and (b) a CVS web-client to some old project works in CVS.Among the latter was e.g. old code to generate manual pages for the proprietary fglrx graphics driver, stuff that laid there untouched for many years. So I guess, it was about time to finally remove it :)The subversion web-client gave public access to some packaging work I do for the Debian GNU/Linux distribution, e.g. for the cvsweb, gtypist packages and some non-official packaging work. For the official packages I plan to move the files into the collab-maint web space and adjust the packages control files accordingly. Everything else will be hosted non-publicly in the future. I still intend to move stuff, that turns out to be useful for more people, to public places like github and Co. Update 17.11.2014: cvsweb, gurlchecker and gtypist have been moved to collab-maint. debian.wgdd.deI used this site to describe my usage of Debian GNU/Linux on the hardware I own ... laptop, servers etc. I wrote a few HOWTOs and provided a link collection with useful links. You can still find all of this using the archive.org service. I also had a repository up and working, especially to provide bluefish packages for users of Debian stable and Ubuntu. Half a year ago I dropped the Ubuntu build environments and packages and moved the Debian stable backports to official places. This effectively emptied the repository and left only the wgdd-archive-keyring package in place. So, there is no real need for a public repository anymore and the linklist probably got outdated too. All in all, I decided to stop this service (maybe I'll forward the site to here later :)).If you see an error regarding the debian.wgdd.de URL running apt-get or aptitude, then there is a reference to this site in /etc/apt/sources.list or /etc/apt/sources.list.d/*, which can be safely removed. Further you should get rid of the wgdd-archive-keyring package:
apt-get autoremove --purge wgdd-archive-keyring
... or the repository key:
apt-key del E394D996
What elseIn case you need any content from the mentioned services, just let me know.

15 September 2014

Gergely Nagy: Looking ahead

A little more than a year ago, I started working as a syslog-ng OSE developer full-time. That was a tremendously important milestone in my career, as one of the goals I wanted to achieve in life - to work on free software for a living - became a reality. Rocket boots were fired up, and we accomplised quite a lot in the past year, and I'm very, very proud of the work we did - we, the whole community. I enjoyed every bit of it, but as it turns out, some of the other desires I wish to pursue, and new challenges I am looking for, will lead me in a different direction. At the end of August, I handed in my resignation, and the past friday was my last work day at BalaBit.There are a few questions that will likely be asked, and I'll try my best to answer them beforehand. Questions such as: What will happen to syslog-ng?, Who will be the new maintainer?, How does this affect your Debian work?, and so on, and so forth.syslog-ngWhat will happen to syslog-ng?After careful consideration, with the syslog-ng team at BalaBit, we decided that they will take over OSE-related responsibilities as well. They will do releases, they will engage the people on GitHub and the mailing list, they will take care of the @sngOSE twitter account.During the past few months, we've been working on pushing the team closer to the community: we moved issues to GitHub, the team submitted pull requests, and in general, we moved closer to each other in every possible way.While they may not have the open source maintainer experience I had, they are all capable folk, and will quickly learn on the job. The team taking over also has the advantage of having more manpower, and better collaboration between the premium edition and OSE.Who will be the syslog-ng maintainer?There will be no single bottle neck. This has both good and bad implications, but I believe the good ones outweigh the disadvantages.What about the roadmap? When will 3.6.1 happen?The roadmap is already laid out for 3.6, but it is the team's judgement when it will be released. The latest 3.6.0beta2 was released together with the team, too. I expect there will be delays (I planned 3.6.1 to be released on September 27th), but not much; a few weeks, perhaps.What will your involvement in syslog-ng development be?With changing jobs, my involvement will drastically decrease. I will remain a syslog-ng user, and I will continue packaging it for Debian and Ubuntu, and will keep my unofficial repository running. I do not see myself contributing much, perhaps bug reports, opinions and an occasional idea.Nevertheless, my expertise is still considerable, and I will help the team and the community in whatever way I can - but I will be severely time constrained.DebianWhile BalaBit were very free software friendly, and allowed me to work on Debian tasks from time to time, my other activities took up an increasingly big chunk of my paid time, and I had to cut back a little. At the new job, things will be a bit different: while I won't have as much time to do Debian work on the job, I will have a lot more free time, in which I hope to do more for Debian.I will - as promised above - continue maintaining syslog-ng, and all other packages I currently maintain. Apart from those, I wish to make myself more useful within the Clojure and Hy teams.MiscellaneousWhere to are you moving?This is something I will answer in due time. For now, I will have two weeks off, which I wish to spend my way, picking up projects I neglected for far too long.

2 April 2014

Tassia Camoes Araujo: Mini-Debconf Barcelona videos now available

Hello world!!! For those who were impatiently waiting for the Mini-Debconf Barcelona videos, there you go, enjoy it! We ll probably have subtitles and some late slides soon, so come back after a while. Thank you very much for all those who make this adventure possible! If you also want to thank the videoteam, the orgateam, Debian Women, or the Universe, for converging and bringing us together in Barcleona, please do it! Just for the records, it was a great success in terms of women participation. As we didn t collect gender information at the registration, it is hard to make a clear comparison with previous Debconfs. Since 2007, the rate of non-male participants ranged from 13% to 17%. For this Mini-Debconf, the orga team did the gender identification per name and found a non-male rate of 36%. Again, since the methods were not the same we cannot safely compare, but still, I think it s worth it to make this info public ;-) The most important thing to save from this experience is that we were around 160 human beings together, sharing common goals, in a lovely and warm place, with kids around, baby trollers on the stage, painting table in the patio yes it was fun!
BCN group photo - first try

BCN group photo first try

BCN group photo - second try

BCN group photo second try

Last but not least, now we need to gather information for a final report, so if you can help, please speak up! Hope to see you all soon!

31 March 2014

Francesca Ciceri: Random notes from MiniDebconf Barcelona 2014

First of all, let me say this: Barcelona MiniDebconf was awesome. So the most important part of this post is a very big thank you to the organizers, volunteers, speakers, sponsors and attendees. Now, down to business. Day 0 Day 0 was a day of travelling, more or less: the Italian Cabal (me, Enrico, Elena and Diego) left in the early morning from Varese (where we had an epic Munchkin Apocalypse game the night before), and we travelled by car across Liguria and France and finally reached Barcelona in the evening. (Boring! Even more if you don't drive. I hate long trips by car) We reached Barcelona in time for the pre-registration event at Falstaff Bar, eat a great Falafel and hugged lots of people. Day 1 To me the highlight of Day 1 was to finally meet in person Laura Arjona, Spanish translator, publicity volunteer, pump.io and mediagoblin contributor. We had decided to have a workshop together about translations and we spent the morning more or less tweaking our presentation and chatting :). So, save for Elena's talk on 3d printing in Debian ("The Universal OS: now making tabletop games and cookie cutters!" -- which was great!) I missed all the talks in the morning. But thanks to the always amazing videoteam, I'll be able to watch them later, when the recordings will be published :). For me, the best talk of the day - well, no, actually of the conference! - was that afternoon: Ana Isabel Carvalho told us about the Libre Graphics Magazine project. Libre Graphics Magazine is a well written and designed magazine at the crossroad between Free Software tools and ideals and graphic arts and design. A crossroad not very much frequented, I'd say. But then, maybe I'm wrong: it's not that graphic artists don't use Free Software tools, it's more like the one who do are invisible.
This is one purpose of a Libre Graphics Magazine: to serve as a catalyst for discussion, to build a home for the users of Libre Graphics software, standards and methods. In such a magazine, we may unite all our previously disparate successes, all the successes which have, until now, stood alone as small examples, disjointed from the larger community. We have the opportunity to elevate the discourse around Libre Graphics as a professionally viable option, to raise previously unmentioned issues and to push forward the conception of just what Libre Graphics can produce.
If you are even only vagued interested in typefaces, fonts, design and graphic art take a look at the magazine: it's CC-BY-SA licensed and you can download it for free, or buy a paper copy (which is amazing, really!). And it's not just about graphic arts: if you skim over the titles of the issues, you can find that they've talked about things like "Localisation/Internationalization", "Use Cases and Affordances" and, my favourite, "Gendering F/LOSS". On a similar topic, Siri's talk about "Why aren't more designers using Debian or working for Debian?" tried to shed a light on the difficult relationship between Free Software tools and graphic artists. These are the voices we need to listen to if we want to bring more graphic artists to Debian, and $deity knows that Debian needs them a lot :). After Solveig's talk about bug triaging and Miriam's one on packaging, it was time for the l10n workshop. I think it went well: we tried to briefly explain the translation workflow in Debian, and to translate together with the audience a po-debconf message. It wasn't maybe enough to complete and submit a translation, but hopefully it gave the audience an idea about how to do it. The day ended with a party for Debian Women 10th anniversary. And the cake wasn't a lie, beside being very good. Day 2 This, I'll remember as "the day I exited my comfort zone". Ok, I'm making a bit of fuss about it, but it was my first talk in English all alone. I spoke about the non-uploading DD process and how to keep your (and others') sanity in a big community project (slides here). I think it's very important to remind people that not all DDs are coding persons. And you don't need to be a developer to love Debian, contribute to it and become an official member of the project. But writing this presentation was for me also the occasion to take stock of my experience in Debian so far: in that talk slipped many of my demons, as impostor syndrome or overcommitment. But all the things I said are more or less, common sense - nothing new! - and lesson learned on the road: it's been now 2 years as DD and 4 as contributor. I'm pretty sure it's thanks to the special conditions of this conference (only speakers identifying themselves as female, a safe and very friendly environment) that I had the courage to give a talk. So the conference was a complete success on that regard, too. In the afternoon I was able to do one of the things I love: videoteam duty. Though I convinced Riccio to switch roles and to give me the camera: my experience in directing during last DebConf left me a bit scarred. Special mention for Laura speaking of pump.io and MediaGoblin and Solveig of Tails during the "Lightning Talks" and the people from LelaCoders during the "Bits from Local Communities" session. The Day(s) after: a Debian Contributors hackathon In my experience a measure of a conference's success is the burst of activity in pet projects just afterwards. In this, also, Barcelona MiniConf was a success: during the weekend, Enrico, Laura and I had the chance to talk together about Debian Contributors and make some plans.
So, as soon as we got in Italy again, I took possession of Enrico's couch for a couple of days and we did a little bit of hacking on Contributors. For my part, I mostly worked in trying to add more data sources to the site: my (not so) secret agenda is to map most of the non-coding contributions. That basically means: translators, publicity editors, event organizers and volunteers, etc. Being in the same room as Enrico, gave me the chance to ask him how to add data sources and to test the existing code (we spot a little problem in the prototype for svn repository mining he made a while ago). At the end of the hackathon, I had managed to: Please note that if you have contributed to one of the repo above and you are not listed there, it means that the automatic recognition of your email address didn't work. We still need to implement a manual interface for the recognition of email addresses: patches are very much welcome! Meanwhile Enrico: If you care about recognition of diverse contributions in Debian, help us: read the project todo list and subscribe to the low traffic mailing list or browse the project Git repository.

20 March 2014

Laura Arjona: Hello, world! I m a Debian contributor

My blog is included now in Planet Debian, so this post is to say hello :) And what can be better to explain who I am, that saying that I am a Debian Contributor? What does Debian Contributor mean? There are lots of tasks that people accomplish to make Debian better, and Debian Contributor is a new hat to credit any participation in the Debian Project, of any kind (even if the contributor is not a full-member of Debian). The idea was started and promoted by Enrico Zini. In less than one year, a website with a list of Debian Contributors, data source schemes and much more pieces have been designed. The site is working, and evolving, according to a roadmap. There are some working data sources already, some other proof of concept , pending to adopt or refine. There are much more kind of contributions that still have no data source created. Everybody can help on that! An example: my contributions I m listed now as wiki editor (at least, I edit once a month when doing my spam fighting tasks), website committer (I upload translations to the Spanish part of the website), and publicity (I ve sent some news to Debian Project News from time to time). Having your name in a list may seem a small thing, but happiness is made of small things!! The website committers data source is one of the proof of concepts and it s not being updated recently. I hope that somebody in the www team takes care of that oh wait, I m part of the www team I m part of the Spam Fighting Force too, but the data source tracks the spam reports reviewers, and I don t do that, I only report spam. That s why I m not there (yet). I m also a translator, member of the Spanish team. Translators are already credited in the translation coordination pages (for example this one), but we want to include them in the main list of Debian Contributors too. That means modifying the bot that tracks the mailing lists. I have joined the i18n team and I hope I can understand how it works and try to propose the required changes. If in the meanwhile, other person comes with the solution, the better! I have given a talk (and two lightning talks) in the last MiniDebConf Barcelona 2014. Most of the speakers in Debian events are available via the Debconf website, for example: http://penta.debconf.org/dc13_schedule/speakers.en.html I m sure a data source can be created somehow! More examples of data sources As I said, these are only examples (it s not about crediting my contributions, but everybody s). So if you explore the already managed data sources, you can find the Debian Sysadmin and Security teams, package maintainers, package uploaders to FTP sites, committers to several source code repositories of Debian-related software, people helping mentoring other people in Debian, people helping in the New Member process, people subtitling talks But we need more!! Please have a look at DcSiteDevel where you can learn how to create a data source and the tools for extracting information in an easy way. If each Debian Team cares of their own data source (which allows to credit their members) it is a sustainable way and also improves those Team s visibility (and new contributions, probably!). Identity management The last updates to Debian Contributors project are about identity management. Right now the site only shows data of contributors having a @debian.org account or an account in Alioth (the Debian software forge, used for many subprojects). It also uses the email field to merge contributions made in areas where you login with other kind of user (e.g. the wiki). This is a great step towards crediting people that are not full Debian members (yet), if they want their names to appear in the Debian Contributors list (they can opt-out too, or show/hide each contribution separately). So, what next? There is a Roadmap and a TODO list. But nobody as individual can know all the roles and actual work in the Debian project. It s better that each team cares about how to send their contribution info to the site. There are some mining tools available to extract for example names of committers in source code repositories, or senders in mailing lists. Not much code is required. Please think about it and join the wave of making Debian diversity skills more visible!
Filed under: Interesting sites, My experiences and opinion, Uncategorized Tagged: Communities, Contributing to libre software, data mining, Debian, Developer motivations, English, libre software, Project Management, statistics, translations

5 March 2014

H ctor Or n Mart nez: Debian build system

There are many ways to build Debian distributions from source. None of them are trivial enough for beginners and all of them require complex setups. For instance, the Debian official setup uses the following software components: Around the core components, there are other software components needed to run the Debian distribution: Thanks to Debian ftp-master and buildd team, all software is built for several architectures and several ports. Most Debian infrastructure is managed and maintained by Debian System Administration team.

Simplified Debian build infrastructure

Debian Wiki has been growing different random pages trying to ease the setup and configuration problems concerning to Debian build system infrastructure. The above infrastructure barely documents what it is involved to create and generate Debian unstable ( sid ) distribution suite. In order to produce Debian stable distribution suite, there are software transitions to happen and yet another Debian team gets involved in the process, our beloved Debian release team. Once distribution reaches its maturity level and gets released as in its stable version, it gets updates also lead by release team and security related updates, which yet another team is responsible for them, the Debian security team. and there are lots of Debian teams doing many other things you might enjoy, have a byte

4 March 2014

H ctor Or n Mart nez: Debian build system

There are many ways to build Debian distributions from source. None of them are trivial enough for beginners and all of them require complex setups. For instance, the Debian official setup uses the following software components: Around the core components, there are other software components needed to run the Debian distribution: Thanks to Debian ftp-master and buildd team, all software is built for several architectures and several ports. Most Debian infrastructure is managed and maintained by Debian System Administration team.

Simplified Debian build infrastructure

Debian Wiki has been growing different random pages trying to ease the setup and configuration problems concerning to Debian build system infrastructure. The above infrastructure barely documents what it is involved to create and generate Debian unstable ( sid ) distribution suite. In order to produce Debian stable distribution suite, there are software transitions to happen and yet another Debian team gets involved in the process, our beloved Debian release team. Once distribution reaches its maturity level and gets released as in its stable version, it gets updates also lead by release team and security related updates, which yet another team is responsible for them, the Debian security team. and there are lots of Debian teams doing many other things you might enjoy, have a byte

28 February 2014

Michael Prokop: Full-Crypto setup with GRUB2

Update on 2014-03-03: quoting Colin Watson from the comments:
Note that this is spelled GRUB_ENABLE_CRYPTODISK=y in GRUB 2.02 betas (matching the 2.00 documentation though not the implementation; not sure why Andrey chose to go with the docs).
Since several people asked me how to get such a setup and it s poorly documented (as in: I found it in the GRUB sources) I decided to blog about this. When using GRUB >=2.00-22 (as of February 2014 available in Debian/jessie and Debian/unstable) it s possible to boot from a full-crypto setup (this doesn t mean it s recommended, but it worked fine in my test setups so far). This means not even an unencrypted /boot partition is needed. Before executing the grub-install commands execute those steps (inside the system/chroot of course, adjust GRUB_PRELOAD_MODULES for your setup as needed, I ve used it in a setup with SW-RAID/LVM):
# echo GRUB_CRYPTODISK_ENABLE=y >> /etc/default/grub
# echo 'GRUB_PRELOAD_MODULES="lvm cryptodisk mdraid1x"' >> /etc/default/grub
This will result in the following dialog before getting to GRUB s bootsplash:

Michael Prokop: State of the art Debian/wheezy deployments with GRUB and LVM/SW-RAID/Crypto

Moving from Lilo to GRUB, using LVM as default, etc throughout the last years it was time to evaluate how well LVM works without a separate boot partition, possibly also on top of Software RAID. Big disks are asking for partitioning with GPT, just UEFI isn t my default yet, so I m still defaulting to Legacy BIOS for Debian/wheezy (I expect this to change for Debian/jessie and according hardware approaching at my customers). So what we have and want in this demonstration setup: System used for installation:
root@grml ~ # grml-version
grml64-full 2013.09 Release Codename Hefeknuddler [2013-09-27]
Partition setup:
root@grml ~ # parted /dev/sda
GNU Parted 2.3
Using /dev/sda
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) mklabel gpt
(parted) mkpart primary 2048s 4095s
(parted) set 1 bios_grub on
(parted) name 1 "BIOS Boot Partition"
(parted) mkpart primary 4096s 100%
(parted) set 2 raid on
(parted) name 2 "SW-RAID / Linux"
(parted) quit
Information: You may need to update /etc/fstab.
Clone partition layout from sda to all the other disks:
root@grml ~ # for f in  b,c,d  ; sgdisk -R=/dev/sd$f /dev/sda
The operation has completed successfully.
The operation has completed successfully.
The operation has completed successfully.
Make sure each disk has its unique UUID:
root@grml ~ # for f in  b,c,d  ; sgdisk -G /dev/sd$f
The operation has completed successfully.
The operation has completed successfully.
The operation has completed successfully.
SW-RAID setup:
root@grml ~ # mdadm --create /dev/md0 --verbose --level=raid5 --raid-devices=4 /dev/sd a,b,c,d 2
mdadm: layout defaults to left-symmetric
mdadm: layout defaults to left-symmetric
mdadm: chunk size defaults to 512K
mdadm: size set to 1465004544K
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.
root@grml ~ #
SW-RAID speedup (system dependent, YMMV):
root@grml ~ # cat /sys/block/md0/md/stripe_cache_size
256
root@grml ~ # echo 16384 > /sys/block/md0/md/stripe_cache_size # 16MB
root@grml ~ # blockdev --getra /dev/md0
6144
root@grml ~ # blockdev --setra 65536 /dev/md0 # 32 MB
root@grml ~ # sysctl dev.raid.speed_limit_max
dev.raid.speed_limit_max = 200000
root@grml ~ # sysctl -w dev.raid.speed_limit_max=9999999999
dev.raid.speed_limit_max = 9999999999
root@grml ~ # sysctl dev.raid.speed_limit_min
dev.raid.speed_limit_min = 1000
root@grml ~ # sysctl -w dev.raid.speed_limit_min=100000
dev.raid.speed_limit_min = 100000
LVM setup:
root@grml ~ # pvcreate /dev/md0
  Physical volume "/dev/md0" successfully created
root@grml ~ # vgcreate homesrv /dev/md0
  Volume group "homesrv" successfully created
root@grml ~ # lvcreate -n rootfs -L4G homesrv
  Logical volume "rootfs" created
root@grml ~ # lvcreate -n bootfs -L1G homesrv
  Logical volume "bootfs" created
Check partition setup + alignment:
root@grml ~ # parted -s /dev/sda print
Model: ATA WDC WD15EADS-00P (scsi)
Disk /dev/sda: 1500GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number  Start   End     Size    File system  Name                 Flags
 1      1049kB  2097kB  1049kB               BIOS Boot Partition  bios_grub
 2      2097kB  1500GB  1500GB               SW-RAID / Linux      raid
root@grml ~ # parted -s /dev/sda unit s print
Model: ATA WDC WD15EADS-00P (scsi)
Disk /dev/sda: 2930277168s
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number  Start  End          Size         File system  Name                 Flags
 1      2048s  4095s        2048s                     BIOS Boot Partition  bios_grub
 2      4096s  2930276351s  2930272256s               SW-RAID / Linux      raid
root@grml ~ # gdisk -l /dev/sda
GPT fdisk (gdisk) version 0.8.5
Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present
Found valid GPT with protective MBR; using GPT.
Disk /dev/sda: 2930277168 sectors, 1.4 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): 212E463A-A4E3-428B-B7E5-8D5785141564
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 2930277134
Partitions will be aligned on 2048-sector boundaries
Total free space is 2797 sectors (1.4 MiB)
Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048            4095   1024.0 KiB  EF02  BIOS Boot Partition
   2            4096      2930276351   1.4 TiB     FD00  SW-RAID / Linux
root@grml ~ # mdadm -E /dev/sda2
/dev/sda3:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 7ed3b741:0774d529:d5a71c1f:cf942f0a
           Name : grml:0  (local to host grml)
  Creation Time : Fri Jan 31 15:26:12 2014
     Raid Level : raid5
   Raid Devices : 4
 Avail Dev Size : 2928060416 (1396.21 GiB 1499.17 GB)
     Array Size : 4392089088 (4188.62 GiB 4497.50 GB)
  Used Dev Size : 2928059392 (1396.21 GiB 1499.17 GB)
    Data Offset : 262144 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : c1e4213b:81822fd1:260df456:2c9926fb
    Update Time : Mon Feb  3 09:41:48 2014
       Checksum : b8af8f6 - correct
         Events : 72
         Layout : left-symmetric
     chunk size : 512k
   Device Role : Active device 0
   Array State : AAAA ('A' == active, '.' == missing)
root@grml ~ # pvs -o +pe_start
  PV         VG      Fmt  Attr PSize PFree 1st PE
  /dev/md0   homesrv lvm2 a--  4.09t 4.09t   1.50m
root@grml ~ # pvs --units s -o +pe_start
  PV         VG      Fmt  Attr PSize       PFree       1st PE
  /dev/md0   homesrv lvm2 a--  8784175104S 8773689344S   3072S
root@grml ~ # pvs -o +pe_start
  PV         VG      Fmt  Attr PSize PFree 1st PE
  /dev/md0   homesrv lvm2 a--  4.09t 4.09t   1.50m
root@grml ~ # pvs --units s -o +pe_start
  PV         VG      Fmt  Attr PSize       PFree       1st PE
  /dev/md0   homesrv lvm2 a--  8784175104S 8773689344S   3072S
root@grml ~ # vgs -o +pe_start
  VG      #PV #LV #SN Attr   VSize VFree 1st PE
  homesrv   1   2   0 wz--n- 4.09t 4.09t   1.50m
root@grml ~ # vgs --units s -o +pe_start
  VG      #PV #LV #SN Attr   VSize       VFree       1st PE
  homesrv   1   2   0 wz--n- 8784175104S 8773689344S   3072S
Cryptsetup:
root@grml ~ # echo cryptsetup >> /etc/debootstrap/packages
root@grml ~ # cryptsetup luksFormat -c aes-xts-plain64 -s 256 /dev/mapper/homesrv-rootfs
WARNING!
========
This will overwrite data on /dev/mapper/homesrv-rootfs irrevocably.
Are you sure? (Type uppercase yes): YES
Enter passphrase:
Verify passphrase:
root@grml ~ # cryptsetup luksOpen /dev/mapper/homesrv-rootfs cryptorootfs
Enter passphrase for /dev/mapper/homesrv-rootfs:
Filesystems:
root@grml ~ # mkfs.ext4 /dev/mapper/cryptorootfs
mke2fs 1.42.8 (20-Jun-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=128 blocks, Stripe width=384 blocks
262144 inodes, 1048192 blocks
52409 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=1073741824
32 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736
Allocating group tables: done
Writing inode tables: done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done
root@grml ~ # mkfs.ext4 /dev/mapper/homesrv-bootfs
mke2fs 1.42.8 (20-Jun-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=128 blocks, Stripe width=384 blocks
65536 inodes, 262144 blocks
13107 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=268435456
8 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376
Allocating group tables: done
Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done
Install Debian/wheezy:
root@grml ~ # mount /dev/mapper/cryptorootfs /media
root@grml ~ # mkdir /media/boot
root@grml ~ # mount /dev/mapper/homesrv-bootfs /media/boot
root@grml ~ # grml-debootstrap --target /media --password YOUR_PASSWORD --hostname YOUR_HOSTNAME
 * grml-debootstrap [0.57] - Please recheck configuration before execution:
   Target:          /media
   Install grub:    no
   Using release:   wheezy
   Using hostname:  YOUR_HOSTNAME
   Using mirror:    http://http.debian.net/debian
   Using arch:      amd64
   Important! Continuing will delete all data from /media!
 * Is this ok for you? [y/N] y
[...]
Enable grml-rescueboot (to have easy access to rescue ISO via GRUB):
root@grml ~ # mkdir /media/boot/grml
root@grml ~ # wget -O /media/boot/grml/grml64-full_$(date +%Y.%m.%d).iso http://daily.grml.org/grml64-full_testing/latest/grml64-full_testing_latest.iso
root@grml ~ # grml-chroot /media apt-get -y install grml-rescueboot
[NOTE: We're installing a daily ISO for grml-rescueboot here because the 2013.09 Grml release doesn't work for this LVM/SW-RAID setup while newer ISOs are working fine already. The upcoming Grml stable release is supposed to work just fine, so you will be able to choose http://download.grml.org/grml64-full_2014.XX.iso by then. :)] Install GRUB on all disks and adjust crypttab, fstab + initramfs:
root@grml ~ # grml-chroot /media /bin/bash
(grml)root@grml:/# for f in  a,b,c,d  ; do grub-install /dev/sd$f ; done
(grml)root@grml:/# update-grub
(grml)root@grml:/# echo "cryptorootfs /dev/mapper/homesrv-rootfs none luks" > /etc/crypttab
(grml)root@grml:/# echo "/dev/mapper/cryptorootfs / auto defaults,errors=remount-ro 0   1" > /etc/fstab
(grml)root@grml:/# echo "/dev/mapper/homesrv-bootfs /boot auto defaults 0 0" >> /etc/fstab
(grml)root@grml:/# update-initramfs -k all -u
(grml)root@grml:/# exit
Clean unmounted/removal for reboot:
root@grml ~ # umount /media/boot
root@grml ~ # umount /media/
root@grml ~ # cryptsetup luksClose cryptorootfs
root@grml ~ # dmsetup remove homesrv-bootfs
root@grml ~ # dmsetup remove homesrv-rootfs
NOTE: On a previous hardware installation I had to install GRUB 2.00-22 from Debian/unstable to get GRUB working.
Some metadata from different mdadm and LVM experiments seems to have been left and confused GRUB 1.99-27+deb7u2 from Debian/wheezy (I wasn t able to reproduce this issue in my VM demo/test setup).
Just in cause you might experience the following error message, try GRUB >=2.00-22:
  # grub-install --recheck /dev/sda
  error: unknown LVM metadata header.
  error: unknown LVM metadata header.
  /usr/sbin/grub-probe: error: cannot find a GRUB drive for /dev/mapper/cryptorootfs.  Check your device.map.
  Auto-detection of a filesystem of /dev/mapper/cryptorootfs failed.
  Try with --recheck.
  If the problem persists please report this together with the output of "/usr/sbin/grub-probe --device-map="/boot/grub/device.map"
  --target=fs -v /boot/grub" to <bug-grub@gnu.org>

24 February 2014

Bits from Debian: Invitation to the MiniDebConf 2014 Barcelona: 15-16 March 2014

Debian Women will hold a MiniDebConf in Barcelona on Saturday 15th and Sunday 16th of March, 2014. Everyone is invited to both talks and social events, but the speakers will all be people who identify themselves as female. This is not a conference about women in Free Software, or women in Debian, rather a usual Debian Mini-DebConf where all the speakers are women. The talks schedule has already been published. It is going to be an exciting event, packed with interesting talks for all audiences, in a beautiful venue, in one of the most famous European cities. Registration is not mandatory, but strongly encouraged, as it helps the event's organisation and logistics. Please, register in the wiki. We are still raising funds to cover the costs of running the conference and to offer travel sponsorship to people who can't pay for it. Please, consider donating any amount you can in our crowd-funding campaign, or contact us if you would like to become a sponsor. The conference organisers want to thank the organisations that have already became sponsors, making this event possible, and specially our Platinum sponsor, Google; our Gold sponsors, C tedra de Programari Lliure - Universitat Polit cnica de Catalunya, Blue Systems and our Silver sponsors, CAtalan LInux Users, CAPSiDE and Fluendo. For more information, visit the website of the event: http://bcn2014.mini.debconf.org

18 February 2014

Margarita Manterola: MiniDebConf 2014 Barcelona

Debian Women is proud to announce that it will hold a MiniDebConf in Barcelona on 15-16 March 2014, where Debian enthusiasts from far and wide will gather to talk about the latest Debian changes and the Debian community, as well as to meet new and old friends. The MiniDebConf is fast approaching, be sure to make your travel arrangements! It is going to be an exciting event, in a beautiful venue, in one of the most famous European cities! Schedule A draft schedule (still subject to some last minute changes) is already available. The list of talks promises two days packed with great talks! If you have an exciting idea for a talk, but missed the deadline, please get in touch with us, we might still have a slot for you. Crowd-funding We are still raising funds to cover the costs of running the conference and to offer travel sponsorship to people who can't pay for it. Please, consider donating any amount you can, everything helps! Sponsorship If you want to attend the conference, but are unable to cover the costs. Please contact us! We don't have a big budget, but we will make our best to help. Naturally, priority will be given to participants who have an accepted talk. About this MiniDebConf We want to have a MiniDebConf with both talks and social events, to which everyone in Debian is invited but where the speakers in the talks are all people who identify themselves as female. We consider this important to The idea behind the conference is not to talk about women in free software, or women in Debian, but rather to make discussion about Debian subjects more inclusive for women. If you agree with this goal, spread the word. Forward this announcement and help us make this event a great success! About our sponsors This event is possible thanks to the help provided by our generous sponsors. In particular, we'd like to thank our platinum sponsor, Google. We hope to see you in Barcelona! For more information, visit http://bcn2014.mini.debconf.org

14 October 2013

Bits from Debian: Ada Lovelace Day: meet some of the "women behind Debian"!

Today is Ada Lovelace Day:
"Ada Lovelace Day is about sharing stories of women - whether engineers, scientist, technologists or mathematicians - who have inspired you to become who you are today. The aim is to create new role models for girls and women in these male-dominated fields by raising the profile of other women in STEM". source
To celebrate, we asked to some of the "women behind Debian" to share their stories with us. Enjoy! Alt Ada Lovelace portrait Ana Guerrero Lopez (ana) Who are you? I'm a 30-something years old geek. I'm from Andalusia, Spain but live in France. What do you do in Debian? I work mostly on my packages, in sponsoring new people's packages and in this very blog you're reading now. I also maintain an unofficial Google+ page about Debian. At $PAID job, I work in an internal Debian distribution so from time to time, I get the opportunity to contribute back some of the stuff we do there. How and why did you start contributing to Debian? I started using Debian around 2003 switching from Mandrake. I was a happy Debian user when the Debian Women project started in the summer 2004. When I saw the project announced, I asked myself why I wasn't contributing to Debian and the rest is history... in a couple of weeks it'll be my 7 years DD-versary! If for some reason you want a longer reply to this question, read here. Beatrice Who are you? I am a PhD student with a degree in Biology. I am a computer fan since my first C64 and I am a self-taught computer geek wanna-be. And I am a bug fan - not software bugs, real bugs :) What do you do in Debian? I work on translations - doing the translation work itself, but also reviewing other translators' work and helping in coordinating the translation effort. How and why did you start contributing to Debian? I started using Linux because I liked the idea of an open source operative system based on collaboration and I began reviewing open source software translations. Since my first Linux system was Debian Potato and I sticked to Debian ever since, it only seemed natural to focus my translation work on Debian. Christine Caulfield Who are you? My name is Christine Caulfield. My day job is principal software engineer at Red Hat working on the cluster infrastructure components corosync & pacemaker. Outside computing I'm a musician and sound engineer. I play violin with lots of technology attached, and love avant garde music. What do you do in Debian? I'm not that active on Debian any more due to pressure of time, and maturity of the packages I work on. I currently maintain the, little-used, DECnet userspace packages and the, even less used I suspect, mopd bootloader. I used to maintain lvm2 for a while but dropped that a few years ago. How and why did you start contributing to Debian? My initial reasons for joining Debian were slightly selfish, to find a home for the DECnet project that I was heavily involved in at the time. I was a keep Debian user and people wanted a distribution where the software was easy to set up. DECnet is quite complicated for users to configure, being a totally independant networking stack to IP and so OS support is needed. Debian seemed like the logical place to make this happen. As mentioned above I got quite involved for a time and maintained other packages too. I picked up lvm2 because I was on the lvm2 dev team at work in Red Hat and as it was a new package at that time I seemed a logical choice. Elena Grandi (valhalla) Who are you? I'm a 30-something years old geek and Free Software enthusiast from Italy. What do you do in Debian? I'm currently maintaining a few packages (2 python modules and a python program) as a sponsored uploader; I'm also slowly looking around for other things to do (by preference technical, but not limited to packaging), with the aim to spend more time contributing to Debian. How and why did you start contributing to Debian? For a while I had being distro-hopping between "fun" distributions (the ones that break now and then) on the desktop while using Debian stable on the home server and in chroots. I was already doing marginal contributions to those distributions, where finding stuff that was missing was easy, but my perception as a stable user was that Debian was already working fine and probably didn't really need any help. Then I started to socialize on IRC with some DDs and DMs, and realized that my perception was superficial and that in reality there were dark holes in the depths of the archive where Evil festered and prospered and... ok, sorry, I got carried away :) Anyway, since I was actually using Debian more and more I decided to start contributing: I read documentation, I attended the useful IRC sessions on #debian-women and decided that it was probably best not to add new stuff, but look for things that I used and that needed help. Then nothing happened for a while, because finding stuff that doesn't work is hard (at least on my mostly textual systems). Then one day I was trying to write a python script that needed to verify gpg signed messages; it had to run on my Debian server, so I was trying to use python-pyme and its documentation was painful to use, while I remembered an earlier attempt using python-gnupg that was much more pythonic, but not available in Debian. In a fit of anger I decided to forgo all of my good intentions and actually add a new package: I checked the sources for problems, packaged, sent it to mentors@d-o, got reviews, fixed problems, resent and finally got sponsored and well, everything started. Francesca Ciceri (madamezou) Who are you? I'm Francesca, a 30-something Italian graduated in Social Sciences. What do you do in Debian? I'm a (non uploading) Debian Developer since 2011 and have been DPN editor, press officer, webmaster for www.debian.org and translator for the Italian l10n team. Recently, due to time constraints, I had to reduce my involvement and now only work on two things: writing/editing articles for bits.debian.org together with Ana Guerrero, and creating subtitles for the DebConf talks, in the DebConf Subs team. How and why did you start contributing to Debian? Basically thanks to the sudden abundance of free time - due to an health problem - and the desire to give something back to this wonderful operating system. After that, I found out that Debian is not only a great OS but also a very special community. Today, some of my dearest friends are people I met through Debian. :) Laura Arjona Who are you? I live in Madrid (Spain), and work as IT Assistant in the Technical University of Madrid (UPM). I'm married and I have a 4-years-old son. What do you do in Debian? In 2012 I started to clean spam and to translate Debian web pages into Spanish. I also follow the work of the web and publicity team, I hope I'll get more involved there too. And of course, I'm in Debian Women :) How and why did you start contributing to Debian? I'm using Debian at work since 2007 (servers), and in my desktops since 2010. I like very much that it is a community distro and I wanted to participate. I was already doing translations in other (small) free software projects, so I began here too. The Debian-Women list, the planet, and people in identi.ca helped me to learn a lot and feel part of the community even when I was not contributing yet. M nica Ram rez Arceda (monica) Who are you? My name is M nica Ram rez Arceda and I am an enthusiast of free software and sharing knowledge cultures: for me it's a kind of philosophy of life. I studied Maths a long time ago but ended up working as a developer for some years. Now I'm working as an IT teacher. What do you do in Debian? Debian is a huge project, so you can help in various scopes. Mainly, I work on packaging, fixing wnpp bug inconsistencies in BTS and helping in spam cleaning of the mailing lists. But I also enjoy doing some non-technical work from time to time: the project I am just now involved is organizing, with the rest of Debian Catalan community, a local team to propose Barcelona as the venue for a minidebconf where all the speakers will be women. How and why did you start contributing to Debian? In 2000 I discovered Free Software world and I fell deeply in love with its philosophy. Since then, I've been trying to do my best in different activities, like spreading the word, giving free courses, helping collectives and friends in technical stuff (from installing Debian to developing some helping apps for them)... but two years ago I was looking forward to join a free software project and I decided to try Debian, since it has been my first and only distro in my day-to-day life for about ten years. So, I wanted to give back Debian all what it had offered to me, but.... I thought I couldn't (hey, Debian is for real hackers, not for you little ant!), but I started to adopt some orphaned packages, do some QA uploads, fix some RC bugs, talk with some Debian Developers that helped me and encouraged me more than I expected, I traveled to my first Debconf... And one thing takes you to the other, and on March 2012 I became a DD. Now, I'm glad to see that everything that frightened me is not so scary :-)

16 July 2013

Joachim Breitner: Real World Haskell Applications

Today I held a lightning talk at the Karlsruhe Functional Programmer s Group, where I presented Real World Haskell applications, i.e. programs that are written in Haskell, are somewhat mature and well-known and have non-programmers as their target group. Here is the (surely subjective) list that I have come up (also see the slides): There is also an impressive list of industrial users that use Haskell without talking about it much, but that does not work well to show off that you can do any kind of application in Haskell. If I missed anything important, let me know!

2 May 2013

Michael Prokop: The #newinwheezy game: Grml packages in Debian/wheezy

Following up on the #newinwheezy game: Debian/wheezy is the first Debian release which ships packages from the Grml system. Grml became an official Debian Derivative and I m very happy that three major projects of Grml found their official way into Debian: As the description states grml2usb is interesting for getting Grml onto a USB device when the dd(1) approach ( dd if=grml.iso of=/dev/sdX ) just isn t flexible enough. grml-debootstrap provides a decent way to install Debian systems from the command line. As its author I might be biased but I ve to mention that it s working so nice that it is in use at several of my customers for automated roll-outs without any worries at all, and I got reports from other companies that they are very happy users of it as well. Finally the grml-rescueboot packages provides a very simple and nice way to boot a rescue system from within GRUB (short version: throw a Grml ISO to /boot/grml/, run update-grub and be done). PS: Thanks everyone for joining the #newinwheezy game over at planet.debian.org. :)

29 April 2013

Michael Prokop: The #newinwheezy game: new forensic packages in Debian/wheezy

Debian/wheezy includes a bunch of packages for people interested in digital forensics. The packages maintained within the Debian Forensics team which are shipped with the upcoming Debian/wheezy stable release for the first time in a Debian release are: Join the #newinwheezy game and present packages which are new in Debian/wheezy.

9 March 2013

Francesca Ciceri: Female participation in Debian (reloaded)

This dent by M nica Ram rez Arceda finally gave me the energy to update the statistics about female participation in Debian, first published by Margarita Manterola back in 2010. So, here the new graph updated to February 2013. Female participation in Debian graph The necessary caveat is that is not easy (at least for me) to tell the gender just based on first names, so the actual numbers may be higher. In addition, you should be warned that there are some contributions not easily traceable: it's the case of translators, graphic artists and documentation writers, as well as event organizers. For instance, Beatrice Torracca doesn't appear in these stats, and she is the co-coordinator of the Italian translation team and a long term Debian (and free software in general) contributor. There was a proposal to better track non packaging contributors, but no news on that front yet (at least that I'm aware of). Sources: You can find the data used as well as the updated graph here. TODO:

21 August 2012

Alexander Pashaliyski: Google Summer of Code 2012 Clang support for build services in Debian Final Result

In the process of installing and configuring wanna-build, I had lots of problems. Because there was almost no documentation on the subject (and some of the existing was outdated), I had to figure out how everything works by reading the source code and asking people on IRC and mailing lists. I use the most recent version (from git) and since it s not stable, there were some problems and bugs. I fixed several bugs in order to set the system up. I also documented every step so now there s documentation on everything I have done. I uploaded the documentation on installing and configuring wanna-build [0]. I installed and configured reprepro on a local machine in order to use it with wanna-build. In the process I made step-by step setup instructions [1]. I installed and configured sbuild. The lastest version of sbuild contains filters but they don t work. Their purpose is to keep only the necessary variables in the environment, but apparenly the configuration file which should set them is never read. That leaves the filter list empty which causes the environment to be empty. I had to disable them to keep my work going. I installed buildd. I had some problems setting it up there was a bug in the scripts related to parsing the configuration file. I was pointed to a helpful bug report on the topic and with some additional changes I eventually got it working. In the process I had to read some of the source code of buildd. I made a patch that fixes a bug in the manual page of reprepro and got it approved. I contacted the Alioth admin team via the web site, requesting to use their server for my project Clang for build services . I got approved and I created a git repository where I m now uploading my work. The repository that I use to upload changes to sbuild and buildd is [2]. I add the name of the used compiler to the control file and to the name of the deb package. I wrote a function for parsing the debian/rules file looking for hard-coded compiler there. If there is such compiler, which is different from the one chosen as a default, the building proccess exits with a clear error. I made changes to wanna-build, dpkg, sbuild and buildd to fix bugs in them and make it possible to use a compiler different from gcc for the build process.
After my work a build compiler can be set easily. If a package can not be built with that compiler, the build process fails with a clear error. I also modified the pgstatus web interface to display build logs and create reports on user request. I wrote complete documentation about setting up wanna-build, reprepro and buildd ([0] and [1]). This makes a fully functional buildd infrastructure. Now we have a full step by step tutorial. Me and Sylvestre Ledru who was my great mentor, modified the buildd daemon so it will be possible to be run several instances of it for different architectures on a single physical server. I made a lot of changes to pgstatus this is a web interface to the wanna-build database. I had to add some features and fix several problems. In the process I read some of the code of pgstatus and buildd-tools. Now I have a better understanding of how they work. I added a new feature in pgstatus that allows the user to create a full report of the state of all builds. When a designated reports page is opened, a table with the current state of the builds is shown, together with a list of all the reports saved to files for the current distribution and architecture. The reports can be downloaded directly from the
interface. They are stored in JSON format. The log files were not accessible from pgstatus because of lack of implementation and bugs. I created a script that reads emails sent by wanna-build and extracts log files from them. I made changes to wanna-build so that it feeds the database with the necessary information about package history. I updated the wanna-build patch to include these changes. As a result the buildd logs can now be opened from the web interface of pgstatus. I applied a patch that fixes bug [3] so that config_schroot.sh (this script installs and sets the default compiler) starts with the necessary privileges. I added a new feature to sbuild that allows scripts to be run right after the dependencies are installed in the chroot and before the actual package starts building. Now anybody can configure sbuild to run such a script by modifying the appropriate configuration file. By adding the following lines: $external_commands =
chroot-pre-build-commands => [
['/root/purge-gcc.sh'],
],
; The script /root/purge-gcc.sh is run by sbuild in the chroot. This script can be necessary to set everything in the environment so that packages are built exactly as needed. Me and my mentor used this to replace gcc with a script that removes all gcc-specific packages so that we can be sure whether a package uses clang or not. I have a fully functional buildd infrastructure installed on my development system. In order to get it running, I made several patches: dpkg.patch The main patch with a lot of changes related to compiler support for building.
wanna_build.patch Fixes bugs and makes wanna-build feed the database with package history information.
sbuild.patch Fixes bugs and work-arounds the problem that some sbuild scripts are called without reading the configuration file of sbuild. Adds configuration option that allows scripts to be run right after the dependencies are installed in the chroot and before the actual package starts building.
pgstatus.patch Fixes bugs and adds reports page to pgstatus. Me and my mentor contacted the Debian wanna-build team to connect our new service with Debian. We explained what the current state of the project is and what we need from them. I thank my mentor Sylvestre Ledru for his presentation on DebConf 12 ([4], [5]). He showed our project and it was seen that people are interested in it. A nice resume of his talk has been made by Michael Larabel on Phoronix [6]. I tested everything thoroughly to make sure my changes don t break anything. I am happy to say that the project is now finished. ==== [0] http://wiki.debian.org/DebianWannaBuildInfrastructureOnOneServer
[1] http://wiki.debian.org/SetupBuildServiceForWanna-build
[2] http://anonscm.debian.org/gitweb/?p=pkg-llvm/clang-build.git;a=summary
[3] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=608840
[4] http://sylvestre.ledru.info/blog/media/5/rebuild-debian-presentation-clang-July-10.pdf
[5] http://meetings-archive.debian.net/pub/debian-meetings/2012/debconf12/high/884_Build_Debian_with_another_compiler.ogv
[6] http://www.phoronix.com/scan.php?page=news_item&px=MTEzODQ

31 January 2012

Rapha&#235;l Hertzog: Contributing to the translation of Debian

If you re not into packaging and if you asked how you could help Debian, someone probably suggested that you help to translate it. It s true that translating Debian is essential if we want to make Debian available to everybody on the world. There are many persons who are stuck as soon as they get a message in English, so it s important to aim for 100% coverage in terms of localization. Some vocabulary: localization vs internationalization Internationalization (i18n) is the work that makes it possible to translate messages in a given application. Localization (l10n) is the work of translating messages of said application. So as a translator, you ll be doing localization but some knowledge of internationalization is still useful because it will define how you re supposed to provide the translations. We ll come back to that later. Join your localization team Usually the translation work is shared among multiple translators within a localization team. Check out the Debian International page on www.debian.org to find out instructions for translators for each language. Many teams have a debian-l10n-*@lists.debian.org mailing list used for coordination, feel free to ask questions on those lists when you start (but make sure that you have read the relevant documentation before). Each team has its own workflow, so observe for a while to get used to what s happening before asking your first questions. What is there to translate? The translation of most of the software provided by Debian is not handled by Debian. The Debian translation teams only handle the translation of: Now before contributing to your first translation, I have to come back to internationalization to teach you a few things. In the above list, the projects marked with (*) do use PO files for their translation and the next sections will explain you how to work with those files. Introduction to Gettext The free software community has mostly standardized on a single internationalization infrastructure known as Gettext. With this tool, you re provided a POT file which contains all the translatable strings. It looks like this:
# SOME DESCRIPTIVE TITLE.
# Copyright (C) YEAR Software in the Public Interest, Inc.
# This file is distributed under the same license as the PACKAGE package.
# FIRST AUTHOR <email>, YEAR.
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: dpkg 1.16.1\n"
"Report-Msgid-Bugs-To: debian-dpkg@lists.debian.org\n"
"POT-Creation-Date: 2011-09-23 03:37+0200\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <email>\n"
"Language-Team: LANGUAGE <ll>\n"
"Language: \n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: nplurals=INTEGER; plural=EXPRESSION;\n"
#: lib/dpkg/ar.c:66
#, c-format
msgid "invalid character '%c' in archive '%.250s' member '%.16s' size"
msgstr ""
#: lib/dpkg/ar.c:81 lib/dpkg/ar.c:97 lib/dpkg/ar.c:108 lib/dpkg/ar.c:112
#: lib/dpkg/ar.c:134 utils/update-alternatives.c:1154
#, c-format
msgid "unable to write file '%s'"
msgstr ""
[ ]
The lines starting with #: are comments that indicate the source files where the (English) string is used. This can be useful if you want check the source to have more information about how the string is used. The lines starting with #, contain flags that can be important. If the fuzzy flag is set, the translated string is not used because it must be updated (or at least verified) since the original string evolved. The c-format flags indicates that the string must be a C format string, this has some implications in what s allowed in the string (in particular when it embeds conversion specifier for arguments submitted to printf-like functions). Another thing to note is that the translation of the empty string is used to store some meta-information about the translation itself. Contributing a translation as a PO file When you start a new translation, you copy that POT file to create a PO file for your own language (eg. fr.po for the French language). You replace some template values (identified with the upper case words in the POT file) and you replace all the empty strings on msgstr lines with the translation of the string that appears in the previous msgid line. The result could be something like this:
# translation of fr.po to French
# Messages fran ais pour dpkg (Linux-GNU Debian).
msgid ""
msgstr ""
"Project-Id-Version: fr\n"
"Report-Msgid-Bugs-To: debian-dpkg@lists.debian.org\n"
"POT-Creation-Date: 2011-09-23 03:37+0200\n"
"PO-Revision-Date: 2012-01-16 07:57+0100\n"
"Last-Translator: Christian Perrier <bubulle>\n"
"Language-Team: French <debian-l10n-french>\n"
"Language: fr\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: Plural-Forms: nplurals=2; plural=n>1;\n"
"X-Generator: Lokalize 1.2\n"
#: lib/dpkg/ar.c:66
#, c-format
msgid "invalid character '%c' in archive '%.250s' member '%.16s' size"
msgstr "caract re invalide  %1$c  dans la taille du membre  %3$.16s  de l'archive  %2$.250s "
#: lib/dpkg/ar.c:81 lib/dpkg/ar.c:97 lib/dpkg/ar.c:108 lib/dpkg/ar.c:112
#: lib/dpkg/ar.c:134 utils/update-alternatives.c:1154
#, c-format
msgid "unable to write file '%s'"
msgstr "impossible d' crire le fichier  %s "
[ ]
If there s already a PO file for your language, there might still work to do: there might be strings that have not yet been translated and there might be fuzzy strings which have to be updated strings which were already translated but where the original string has been modified. There are software that can assist you to edit PO files: poedit, virtaal, lokalize, gtranslator. There are also special extensions for vim (packaged in vim-scripts) and for Emacs. Submit the translation for inclusion Once you have a complete PO file, you should submit it for inclusion. Sometimes you will have been granted commit rights to the source code repository so that you can include your translation by yourself. In the other cases, you should submit your translation with a bug report tagged l10n and someone else will include your work in the next release. Depending on the team, the workflow might require a review before the submission. In that case, you usually have to send a call for review on the coordination mailing list. Go ahead! Hopefully those explanations will be enough to get you started. There are many other things to learn but it s good to learn while practicing For example, can you find out why the French translation above changed %c in %1$c ?
Do you want to read more tutorials like this one? Click here to subscribe to my free newsletter, you can opt to receive future articles by email.

No comment Liked this article? Click here. My blog is Flattr-enabled.

1 July 2011

Ian Wienand: Debugging __thead variables from coredumps

I think the old-fashioned coredump is a little under-appreciated these days. I'm not sure when it changed, but I even had to add myself to /etc/security/limits.conf to raise my ulimit to even create one. Anyway, debugging __thread variables from coredumps is a bit of a pain. For the uninitiated, the __thread identifier specifies a thread-local variable, i.e. every thread gets its own copy of the variable automatically. The implementation of this is highly architecture specific. The reason is that TLS entries need to be accessed via a register kept as part of the thread state, and thus every architecture chooses their own register and builds their own ABI. On x86-32, which is very register-limited, you certainly don't want to dedicate a register to a pointer to TLS variables and take it out of operation. Luckily there is the hang-over from the 70's (60's? 50's?) segmentation. Without going into real detail, segment registers can be used to offset into a region of memory based on a look-up of a region descriptor stored in a table. Simple example of gs and the GDT Above, you see a simplified example of the %gs register loaded with the index value 2, and thus when you access %gs:20 what you are saying is "find entry 2 in the global descriptor table (GDT), follow it and offset 20 into that region. The kernel gives each thread its own GDT (i.e. the GDT register is part of the thread-state). Thus __thread variables are stored based on segment offsets and voila thread-local storage. Now, there's a few tricks here. For various reasons, a process can not setup entries in the GDT; this is a privileged operation that must be done by the kernel. There is actually a special system call for threads to setup their TLS areas in the GDT set_thread_area. When a new thread starts, the thread-library and dynamic linker conspire to allocate and load any static TLS data (i.e. if you have a global __thread variable initialised to some value, then every thread must see that value when it starts) and then calls this to make sure the variables are ready to go. After that, the gs register is filled with the index of that GDT entry, and all TLS access goes via it. That, in a nut-shell, is TLS for x86-32. Now, to the problem. Take, for example, the following short program:
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <pthread.h>
int __thread foo;
void* thread(void *in)  
        foo = (int)in;
        printf("foo is %d\n", foo);
        while (1)  
                sleep(10);
         
 
int main(void)  
        pthread_t threads[5];
        int i;
        for (i=0; i<5; i++)  
                pthread_create(&threads[i], NULL,
                               thread, (void*)i);
         
        sleep(5);
        abort();
 
We start a few threads and then abort to make it dump core. But, if you try and examine foo:
$ gdb ./thread core 
GNU gdb (GDB) 7.2-debian
Reading symbols from /home/ianw/tmp/thread/thread...done.
[New Thread 4970]
[New Thread 4975]
[New Thread 4974]
[New Thread 4973]
[New Thread 4972]
[New Thread 4971]
Core was generated by  ./thread'.
Program terminated with signal 6, Aborted.
#0  0xffffe424 in __kernel_vsyscall ()
(gdb) print foo
Cannot find thread-local variables on this target
It seems that gdb doesn't know how to find the value of foo because its not a variable in the usual sense ("this target", in this case, means a coredump). It relies on accessing via the gs register, which relies on the current processes' GDT state, which has since been destroyed. If you care to consult the canonical source of TLS info, you can find out exactly why this is so hard to figure out generically. However, with some work, we can start to figure out the value by hand. A coredump is really a ELF file full of just two things: a bunch of LOAD segments that are just dumps of the process memory regions, and a NOTE section that includes a bunch of notes that the kernel dumps out for us such as the current register state, the process id, the signal that killed us, etc. Here's an example of a core file under readelf
$ readelf --headers ./core 
ELF Header:
  Magic:   7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00 
  Class:                             ELF32
...
  OS/ABI:                            UNIX - System V
...
  Type:                              CORE (Core file)
...
There are no sections in this file.
Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  NOTE           0x0003d4 0x00000000 0x00000000 0x006b4 0x00000     0
  LOAD           0x001000 0x08048000 0x00000000 0x00000 0x01000 R E 0x1000
  LOAD           0x001000 0x08049000 0x00000000 0x01000 0x01000 RW  0x1000
  LOAD           0x002000 0x0804a000 0x00000000 0x21000 0x21000 RW  0x1000
  LOAD           0x023000 0x46015000 0x00000000 0x00000 0x1b000 R E 0x1000
  LOAD           0x023000 0x46030000 0x00000000 0x01000 0x01000 R   0x1000
  LOAD           0x024000 0x46031000 0x00000000 0x01000 0x01000 RW  0x1000
...
You can examine the notes; with readelf we see:
$ readelf --notes ./core 
Notes at offset 0x000003d4 with length 0x000006b4:
  Owner                 Data size      Description
  CORE                 0x00000090      NT_PRSTATUS (prstatus structure)
  CORE                 0x0000007c      NT_PRPSINFO (prpsinfo structure)
  CORE                 0x000000a0      NT_AUXV (auxiliary vector)      
  LINUX                0x00000030      Unknown note type: (0x00000200) 
  CORE                 0x00000090      NT_PRSTATUS (prstatus structure)
  LINUX                0x00000030      Unknown note type: (0x00000200) 
  CORE                 0x00000090      NT_PRSTATUS (prstatus structure)
  LINUX                0x00000030      Unknown note type: (0x00000200) 
  CORE                 0x00000090      NT_PRSTATUS (prstatus structure)
  LINUX                0x00000030      Unknown note type: (0x00000200)
  CORE                 0x00000090      NT_PRSTATUS (prstatus structure)
  LINUX                0x00000030      Unknown note type: (0x00000200)
  CORE                 0x00000090      NT_PRSTATUS (prstatus structure)
  LINUX                0x00000030      Unknown note type: (0x00000200)
This shows 5 notes of type NT_PRSTATUS; it should be little surprise that each of these notes describes the process status of one running thread. When gdb pops up [New Thread 4973] that's because it just hit another note that describes the new thread. To actually make sense of the note, however, we need some other tools that look deeper. The elfutils based tools give us a better description of the various notes in the coredump; more akin to what GDB is interpreting them as. Below I've extracted one thread's info:
$ eu-readelf --notes core
...
  CORE                 144  PRSTATUS
    info.si_signo: 6, info.si_code: 0, info.si_errno: 0, cursig: 6
    sigpend: <>
    sighold: <>
    pid: 4970, ppid: 4960, pgrp: 4970, sid: 4960
    utime: 0.000000, stime: 0.000000, cutime: 0.000000, cstime: 0.000000
    orig_eax: 270, fpvalid: 0
    ebx:           4970  ecx:           4970  edx:              6
    esi:              0  edi:     1176018932  ebp:     0xbfa5b710
    eax:              0  eip:     0xffffe424  eflags:  0x00000206
    esp:     0xbfa5b6f8
    ds: 0x007b  es: 0x007b  fs: 0x0000  gs: 0x0033  cs: 0x0073  ss: 0x007b
  LINUX                 48  386_TLS
    index: 6, base: 0xb6043b70, limit: 0x000fffff, flags: 0x00000051
    index: 7, base: 0x00000000, limit: 0x00000000, flags: 0x00000028
    index: 8, base: 0x00000000, limit: 0x00000000, flags: 0x00000028
What's particularly interesting here is that the note type that was previously unknown (Unknown note type: (0x00000200)) has been resolved for us it is in fact of type NT_386_TLS and is a dump of the GDT entries for the thread. So, if we examine how our function is accessing our TLS variable by disassembling the thread function:
(gdb) disassemble thread
Dump of assembler code for function thread:
   0x08048514 <+0>:    push   %ebp
   0x08048515 <+1>:    mov    %esp,%ebp
   0x08048517 <+3>:    sub    $0x18,%esp
   0x0804851a <+6>:    mov    0x8(%ebp),%eax
   0x0804851d <+9>:    mov    %eax,%gs:0xfffffffc
   0x08048523 <+15>:   mov    %gs:0xfffffffc,%edx
   0x0804852a <+22>:   mov    $0x8048670,%eax
   0x0804852f <+27>:   mov    %edx,0x4(%esp)
   0x08048533 <+31>:   mov    %eax,(%esp)
   0x08048536 <+34>:   call   0x80483f4 <printf@plt>
   0x0804853b <+39>:   movl   $0xa,(%esp)
   0x08048542 <+46>:   call   0x8048404 <sleep@plt>
   0x08048547 <+51>:   jmp    0x804853b <thread+39>
Examining the disassembly we can see that foo is accessed via an offset of -4 from %gs (this is OK, as our limit value is maxed out. See the TLS ABI doc for more info). Now, we can examine gs and see which selector it is telling us to use:
(gdb) print $gs >> 3
$31 = 6
Above we shift out the last 3 bits, as these refer to the privilege level (bits 0 and 1) and if this is a GDT or LDT reference (bit 2). Thus, looking at the GDT descriptor for index 6:
  LINUX                 48  386_TLS
    index: 6, base: 0xb6043b70, limit: 0x000fffff, flags: 0x00000051
we can finally do some maths from the base-address to figure out the value:
(gdb) print *(int*)(0xb6043b70 - 4)
$34 = 3
Thus we have found our TLS value; in this case for thread 3 the value is indeed 3. I caution this is the simplest possible case; other "models" (see the TLS doc, again) may not be so simple to work out by hand, but this would certainly be how you would start.

2 March 2011

Ian Wienand: Pyblosxom Recaptcha Plugin

I think maybe the world has moved on from Pyblosxom and, as my logs can attest, spammers seem to have moved on past Recaptcha (even if it is manual). However, maybe there are some other holdouts like myself who will find this plugin useful. In reality, this will do nothing for stopping spam for that I am currently having good luck with Akismet. However, I like that in displaying the captcha it costs the spammers presumably something to crack it so they can even attempt to submit the comment.

Next.

Previous.